Apparatus and method for transmission, apparatus and method for production, program, and recording medium

ABSTRACT

Transmit data composed by linking to main data representing an image and/or audio accessory information including frame rate information and frame identification information of each frame included in a reference frame period is generated and output. If the main data is reproduced using this transmit data, a variable reproduction speed range is set based on the frame rate information. In accordance with a specified reproduction speed within the variable reproduction speed range, thinning-out or repeating processing is performed on the data of image and/or audio utilizing the frame identification information, thereby making a reproduction speed of the main data variable easily to generate image signals or audio signals.

TECHNICAL FIELD

The present invention relates to an apparatus and method fortransmission, an apparatus and method for reproduction, a program, and arecording medium.

BACKGROUND ART

Conventionally, in generation of image and/or audio contents to be usedin a broadcast, contents (or a content) in which a speed of movement ofan object is partially varied has been often created in order to achieveresults a creator intends.

In this generation of the contents in which the speed of movement of theobject is varied partially, for example, the contents are generated bysetting it at a frame rate higher than a reference frame rate and thenreproduced at the reference frame rate, thereby generating the contentsexpressing the movement of the object slow. Further, the contents aregenerated by setting it at a frame rate lower than the reference framerate and then reproduced at the reference frame rate, thereby generatingthe contents expressing the movement of the object fast. Furthermore, byadjusting a frame rate to be set or a frame rate for reproduction, thespeed of movement of the object can be varied at will.

In such a manner, the creator generates broadcast contents using notonly the contents generated at the reference frame rate but also thecontents having a varied speed of movement of the object, in order toachieve results he or she intends when the contents are reproduced atthe reference frame rate.

Further, a video camera that can contract and expand a time axis togenerate the contents having a varied frame rate in such a manner isdescribed in, for example, Jpn. Pat. Appln. KOKAI Publication No. Hei11-177930.

On the other hand, with an increase in bandwidth and a decrease in costof a communication network, it has been put to practical use to transmitcontents via this communication network interactively. In transmissionof the contents via the communication network, the transmitted contentsare stored in a buffer temporarily and then reproduced, therebyabsorbing variations occurring over the communication network(fluctuations in arrival of data) to continuously reproduce thecontents. Further, with an increase in bandwidth of the communicationnetworks, a larger amount of data can be transmitted.

However, even in this interactive transmission for the contents, as inthe case of broadcasting, the contents generated so that an intendedresult may be achieved when they are reproduced at the reference framerate have been used as the transmit contents. Therefore, even if thereproduction is performed at a desired speed that is different from thecreator's intended speed, a portion of such the contents generated at avaried frame rate cannot be reproduced at the desired speed.

DISCLOSURE OF THE INVENTION

A transmission apparatus related to the present invention comprisestransmit data generation means for generating transmit data by linkingto main data representing an image and/or audio accessory informationincluding information on a frame rate of this main data, andtransmission processing means for performing output processing on thetransmit data via a transmission channel.

A transmission method related to the present invention comprises atransmit data generation step for generating transmit data by linking tomain data representing an image and/or audio accessory informationincluding information on a frame rate of this main data, and atransmission processing step for performing output processing on thetransmit data via a transmission channel.

A reproduction apparatus related to the present invention comprisessetting means for, based on information of a frame rate of transmit datacomposed by linking to main data representing an image and/or audioaccessory information including information of a frame rate of this maindata, setting a variable reproduction range indicating a range of theframe rate of the main data to be reproduced; and reproduction means forreproducing the main data at a frame rate in the variable reproductionrange.

A reproduction method related to the present invention comprises asetting step for, based on information of a frame rate of transmit datacomposed by linking to main data representing an image and/or audioaccessory information including information of a frame rate of this maindata, setting a variable reproduction range indicating a range of theframe rate of the main data to be reproduced, and a reproduction stepfor reproducing the main data at a frame rate in the variablereproduction range.

A program related to the present invention causes a computer to performa transmission method, the method comprising: a transmit data generationstep for generating transmit data by linking to main data representingan image and/or audio accessory information including information on aframe rate of this main data; and a transmission processing step forperforming output processing on the transmit data via a transmissionchannel. It also causes a computer to perform a reproduction method, themethod comprising: a setting step for, based on information of a framerate of transmit data composed by linking to main data representing animage and/or audio accessory information including information of aframe rate of this main data, setting a variable reproduction rangeindicating a range of the frame rate of the main data to be reproduced;and a reproduction step for reproducing the main data at a frame rate inthe variable reproduction range.

A recording medium related to the present invention records main datarepresenting an image and/or sound with accessory information includinginformation of a frame rate of this main data being linked to the maindata.

According to the present invention, to main data representing an imageand/or sound, accessory information including information of a framerate of this main data is linked so that they can be output as transmitdata. It is to be noted that the main data is stored, for example,temporarily, which main data thus stored is read in accordance with aband of a transmission channel to adjust a frame rate of the main data,so that in accordance with this frame rate adjustment, the frame rateinformation included in the accessory information is modified andlinked. This accessory information includes information indicating arecommended reproduction speed of the main data and informationindicating a maximum speed at which the main data can be reproduced.Further, as the accessory information, at least frame rate informationand frame identification information of each frame included in areference frame period are linked to the main data, so that by utilizingthis frame identification information, reading of the main data iscontrolled in accordance with an informed band, thereby adjusting theframe rate of the main data.

In a case where the main data is reproduced using the transmit data inwhich the accessory information is linked to the main data, based oninformation of a frame rate, a variable reproduction range indicating arange of the frame rate of the main data to be reproduced is set, sothat at a frame rate within this variable reproduction range the maindata is reproduced. Further, in a case where the accessory informationincludes information indicating a recommended reproduction speed of themain data and a reproduction speed is not specified by a user, the maindata is reproduced at this recommended reproduction speed. Further, in acase where the accessory information includes information indicating amaximum speed at which the main data can be reproduced, a variablereproduction speed range is set using the information indicating themaximum speed. Further, the accessory information includes frameidentification information of each frame included in the reference frameperiod, so that by thinning out or repeating this main data by utilizingthe frame identification information, the reproduction speed of the maindata is varied.

According to the present invention, to the main data representing animage and/or sound, accessory information including information of aframe rate of this main data is linked and output as the transmit data.Further, when reproducing the main data using this transmit data, basedon the frame rate information included in the accessory information, avariable reproduction range indicating a range of the frame rate of themain data to be reproduced is set, so that at a frame rate in thisvariable reproduction range the main data is reproduced. Therefore, bylinking the accessory information to a portion of the contents, whichcan be reproduced in a range of a speed different from that intended bya creator of the contents, that portion can be reproduced at thisdifferent speed.

Further, the main data is stored temporarily and then read in accordancewith a band of a transmission channel, thereby adjusting a frame rate ofthe main data. Thus, the frame rate of the main data can be adjustedeasily. Further, it is possible to prevent an image or sound from beinginterrupted during reproduction. Furthermore, in accordance withadjustment of the frame rate, frame rate information contained in theaccessory information is modified, so that the accessory informationcorresponding to the main data to be transmitted can be linked to it.

Further, if a reproduction speed including information indicating arecommended reproduction speed of main data is not specified in theaccessory information when the main data is reproduced, the main data isreproduced at a recommended reproduction speed, so that thisreproduction speed thereof can be specified by a creator side of themain data.

Further, information indicating a maximum speed at which the main datacan be reproduced is included in the accessory information as well as avariable reproduction speed range are set using this informationindicating the maximum speed when the main data is reproduced, so thatthe variable reproduction speed range can be regulated by the creatorside of the main data.

Furthermore, as the accessory information, at least frame rateinformation and frame identification information of each frame includedin the reference frame period are linked to the main data, so that byutilizing this frame identification information, a frame rate of themain data is adjusted to generate transmission data. Further, when themain data is reproduced using transmission data, by utilizing the frameidentification information, the main data is thinned out or repeated tovary a reproduction speed of the main data. Therefore, the main data canbe reproduced at a desired speed according to a simple configuration.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram for showing an overall configuration of acontents-transmission system;

FIG. 2 is a diagram for showing a configuration of an image pick-upapparatus;

FIG. 3 is a diagram for showing an operation of adding a sub-framenumber;

FIG. 4 is a diagram for showing another configuration of the imagepick-up apparatus;

FIGS. 5A-5E are diagrams each for showing a (first) relationship ofimage data to accessory information;

FIGS. 6A-6E are diagrams each for showing a (second) relationshipbetween image data and accessory information;

FIG. 7 is a diagram for showing a configuration of an editing apparatus;

FIG. 8 is a diagram for showing a configuration of acontents-transmission apparatus;

FIG. 9 is a diagram for showing a configuration in a case where contentsare transmitted by software;

FIG. 10 is a flowchart for showing contents-transmission processingoperation;

FIG. 11 is a diagram for showing a configuration of acontents-reproduction apparatus;

FIG. 12 is a diagram for showing a configuration in a case wherecontents are reproduced by software;

FIG. 13 is a flowchart for showing a contents-reproduction processingoperation;

FIG. 14 is a diagram for showing an image displayed on acontents-presentation apparatus;

FIG. 15 is a flowchart for showing an operation of setting reproductionprocessing conditions to an image;

FIGS. 16A-16M are diagrams each for showing a (first) image reproductionoperation;

FIGS. 17A-17M are diagrams each for showing a (second) imagereproduction operation;

FIGS. 18A-18M are diagrams each for showing a (third) image reproductionoperation;

FIG. 19 is a flowchart for showing an operation of setting reproductionprocessing conditions to sound;

FIGS. 20A-20E are diagrams each for showing a (first) audio reproductionoperation; and

FIGS. 21A-21E are diagrams each for showing a (second) audioreproduction operation.

BEST MODE FOR CARRYING OUT THE INVENTION

The following will describe the present invention with reference toaccompanying drawings. FIG. 1 shows an overall configuration of acontents-transmission system for transmitting contents (or a content),for example, image and/or audio contents. An image pick-up apparatus 10generates image data having a varied frame rate and links accessoryinformation associated with this image data thereto and then supplies itas materials-data DTm to an editing apparatus 30. Further, when equippedwith an audio input apparatus 20, the image pick-up apparatus 10generates audio data and supplies this audio data also to the editingapparatus 30 as materials-data DTm. It is to be noted that thematerials-data DTm may be supplied not only from the image pick-upapparatus 10 but also from any other appliances.

The editing apparatus 30 performs edit processing by using the suppliedmaterials-data DTm, to generate data which represents images and/orsound desired by an editor. Further, the data representing the imagesand/or sound is provided as main data and, accessory information islinked to this main data to generate contents data DC for transmission,which is supplied to a contents-transmission apparatus 50.

The editing apparatus 30 generates an image signal Svm associated withediting and supplies it to an edited-image display 40. Accordingly, auser can confirm image edit processes, results, etc. using an imagedisplayed on the edited-image display 40. Similarly, it generates anaudio signal Sam associated with editing and supplies it to anedited-audio output apparatus 41. Thus, the user can confirm audio editprocesses, results, etc. using a sound output from the edited-audiooutput apparatus 41.

A contents-transmission apparatus 50 accumulates contents-data DCsupplied from the editing apparatus 30. Further, if receiving a requestfor contents-data from, for example, a contents-reproduction apparatus70, it adjusts a frame rate of the contents-data in accordance with aband of a transmission channel 60, generates transmit data DTc based onthe contents-data after being adjusted in terms of frame rate, andsupplies this transmit data DTc to the contents-reproduction apparatus70 via the wireline or wireless transmission channel 60.

The contents-reproduction apparatus 70 generates an image signal Svz oran audio signal Saz of the contents based on the transmit data DTcsupplied via the transmission channel and supplies it to acontents-presentation apparatus 80. Further, the contents-reproductionapparatus 70 controls reproduction operation of the contents based onthe accessory information.

The contents-presentation apparatus 80 displays an image based on theimage signal Svz or outputs a sound based on the audio signal Saz,thereby presenting the contents.

Regarding linkage, in this context, it may be such a condition that themain data and the accessory information including a frame rateconcerning this main data are linked to each other. For example, even ifthe main data and the accessory information have been transmitted viadifferent transmission channels, they can be correlated with each otheras far as a frame number corresponding to the accessory information iscontained therein. The linkage in the present embodiment includes such acase.

FIG. 2 shows a configuration of an image pick-up apparatus 10. Lightpassing through an image pick-up lens system 11 impinges on an imagepick-up portion 12, so that an image of an object is formed on an imagepick-up surface of an image pick-up device such as a charge coupleddevice (CCD) equipped to the image pick-up portion 12. The image pick-updevice generates imaged charge of the object image by utilizingphotoelectric transfer. Further, the image pick-up portion 12 reads theimaged charge, which has been generated, on the basis of a drive signalCR from a timing generator 142, described later, generates an imagepick-up signal Sp of a frame rate in accordance with the drive signalCR, and supplies it to a camera-processing circuit 131 in asignal-processing portion 13.

The camera-processing circuit 131 performs a various kinds of signalprocessing at a timing synchronized with the image pick-up signal Spbased on a timing signal CT supplied from the timing generator 142. Forexample, the camera processing circuit 131 performs processing to filterout a noise component from the image pick-up signal Sp by utilizingcorrelated double sampling etc., processing to convert the noise-freeimage signal Sp into digital image data, processing to clamp the imagedata, shading correction or defect correction of the image pick-updevice, γ processing or profile compensation processing, Knee correctionprocessing, etc. Further, the camera processing circuit 131 performsvarious kinds of signal processing under processing conditions that arebased on an operation control signal CS supplied from an image pick-upcontrol circuit 141 in a control portion 14 or the like. In such amanner, image data DV obtained as a result of the various kinds ofsignal processing performed at the camera-processing circuit 131 issupplied to an output portion 15.

The timing generator 142 in the control portion 14 generates the drivesignal CR in accordance with the operation control signal CS from theimage pick-up control circuit 141 and supplies it to the image pick-upportion 12 to thus vary a cycle at which the imaged charge is read atthe image pick-up portion 12, thereby regulating a frame rate of theimage pick-up signal Sp to a set frame rate FRs based on an operationsignal PSa from a user interface portion 16. The timing generator 142conducts such control that, by assuming, for example, a frame frequencyof 59.94 Hz or 29.97 Hz in the case of NTSC system or a frame frequencyof 50 Hz or 25 Hz in the case of the PAL system as a frame frequency ofa reference frame rate FRr, a frame rate of the image pick-up signal Spmay be k (which is a positive value but not limited to an integer) timesthe reference frame rate FRr if an operation to make the set frame rateFRs k times the reference frame rate FRr is performed. It is to be notedthat the cycle for reading the imaged charge is varied by, for example,altering a cycle of a read pulse (sensor gate pulse) for moving theimaged charge accumulated in each pixel of the image pick-up device suchas a CCD to a transfer portion, thereby varying the frame rate. Further,in this case, the common data rate (CDR) method may be employed. Byusing the CDR method, valid frame rates can be varied while the framerate of a signal output from the CCD stays unchanged, thereby setting aprocessing rate of the camera-processing circuit 131 etc. constant. ThisCDR method is disclosed in PCT application No. PCT/JP03/00551, filed on2003 Jan. 22.

Further, the timing generator 142 generates the timing signal CTsynchronized with the drive signal CR and supplies it to thecamera-processing circuit 131 and the audio-processing circuit 132.Furthermore, the timing generator 142 generates frame rate informationDM-FRs indicating the set frame rate FRs, which is a frame rate of imagedata DV, and supplies it to the output portion 15. Further, the timinggenerator 142 generates a sub-frame number BN. This sub-frame number BNis a number that enables to be identified each frame included in a frameperiod of the reference frame rate FRr when the set frame rate FRs isset higher than the reference frame rate FRr. This sub-frame number BNis supplied to the output portion 15 as frame identification informationDM-BN.

FIG. 3 is a flowchart for showing an operation of adding the sub-framenumber at the timing generator 142. The timing generator 142 divides anoscillation frequency having, for example, a predetermined frequency andsets a frame period of the reference frame rate FRr and that of the setframe rate FRs so that they can be synchronized with each other, therebygenerating the drive signal CR and a frame reference timing thatindicates a breakpoint of the frame period of the reference frame rateFRr, based on the frame period of the set frame rate FRs.

The timing generator 142 identifies whether the frame reference timingis detected at step ST1. If the frame reference timing is detected, theprocess goes to step ST2. If no frame reference timing is detected, theprocess returns to step ST1.

If the frame reference timing is detected at step ST1 and the processgoes to step ST2, the timing generator 142 initializes the sub-framenumber BN at step ST2 so that the sub-frame number BN is set to, forexample, “0” and the process goes to step ST3.

At step ST3, the timing generator 142 identifies whether the framereference timing is detected during a one-frame period of time lapse ofthe set frame rate FRs starting from a moment of detection of the framereference timing. If no frame reference timing is detected, the processgoes to step ST4 where the timing generator 142 adds “1” to thesub-frame number BN to update it and the process then returns to stepST3. In such a manner, if no frame reference timing is detected during aone-frame period of time lapse of the set frame rate FRs, the sub-framenumbers BN are sequentially assigned every one-frame period of the setframe Fate FRs.

Then, if the frame reference timing is detected before a one-frameperiod of the set frame rate FRs elapses, the process returns to stepST2 to initialize the sub-frame number BN.

Therefore, in each frame period of the reference frame rate FRr, thesub-frame number BN can be added to a frame image in the set frame rateFRs which is provided during this frame period.

To the image pick-up control circuit 141 in the control portion 14 shownin FIG. 2, the user interface portion 16 is connected. If the imagepick-up apparatus 10 switches its operation or varies a frame rate, theuser interface portion 16 generates an operation signal PSa inaccordance with these operations and supplies it to the image pick-upcontrol circuit 141. Further, the user interface 16, if supplied withthe operation signal PSa from an external appliance such as a remotecontroller, not shown, supplies this operation signal PSa to the imagepick-up control circuit 141.

Based on the operation signal PSa from the user interface portion 16,the image pick-up control circuit 141 generates the control signal CS sothat the image pick-up apparatus 10 may operate in accordance with theoperation signal PSa and supplies it to the camera-processing circuit131 and the timing generator 142.

The audio-processing circuit 132 is supplied with an analog audio signalSin from the audio input apparatus 20. The audio-processing circuit 132performs sampling processing on the audio signal Sin based on the timingsignal CT supplied from the timing generator 142 to generate digitalaudio data DA and supplies it to the output portion 15.

The output portion 15 generates accessory information DM including framerate information DM-FRs and frame identification information DM-BN andlinks it to the image data DV and the audio data DA to generatematerials-data DTm and supplies it to the editing apparatus. It is to benoted that by recording in a recording medium the materials-data DTm ora record signal generated on the basis of the materials-data DTm, it ispossible to reproduce the recording medium in which this materials-dataDTm or the record signal generated on the basis of the materials-dataDTm, thereby supplying the materials-data DTm to the editing apparatusvia the recording medium. Further, the accessory information DM maycontain not only information on the set frame rate FRs and the sub-framenumber BN but also information indicating an imaged date/time, imageconditions, image details, etc.

As one example of a method for linking the accessory information DM tothe image data DV or the audio data DA, such an approach may beconsidered for, when the image data DV or the audio data DA iscompressed to generate materials-data DTm as a data stream, insertingthe accessory information DM into the data stream of an image orinserting the accessory information DM into a header of the data stream.

Further, in the case of using an SDI format standardized as SMPTE(Society of Motion Picture and Television Engineers) 259M“Television—10-Bit 4:2:2 Component and 4fsc Composite DigitalSignals—Serial Digital Interface” to transmit non-compressed image dataor audio data, an SDTI format standardized as SMPTE305M“Television—Serial Data Transport Interface (SDTI)” to transmitcompressed image data or audio data, or an SDTI-CP format standardizedas SMPTE326M “Television—SDTI Content Package Format (SDTI-CP)” which isa further restricted version of the SDTI format, the accessoryinformation DM is given as data of an UMID standardized as SMPTE330M“Television—Unique Material Identifier (UMID)” and inserted into asignal in each of the formats. It is to be noted that the method oflinking the accessory information to the image data DV or the audio dataDA is not limited to it but a variety of other methods may beconsidered. Further, linkage may require only that a relationshipbetween one and another can be known by any means, that is, they can belinked to each other. For example, even if they are sent throughdifferent transmission channels, they can be correlated to each other asfar as they are provided with the same UMID, which case is alsocategorized as linkage.

Note here that the above-mentioned image pick-up apparatus 10 varies acycle for reading imaged charge at the image pick-up portion 12 tothereby generate materials-data DTm having a desired set frame rate FRs,so that the set frame rate FRs can be varies continuously. However, ifthe set frame rate FRs needs only to be varied step-wise, thematerials-data DTm having a desired set frame rate FRs can be generatedby thinning out frames. That is, by generating image data DVa having aconstant frame rate higher than the set frame rate FRs and extractingimage data as much as the set frame rate FRs from this image data DVa,it is possible to generate the image data DV having the set frame rateFRs. A configuration in this case is shown in FIG. 4. It is to be notedthat in FIG. 4, components that corresponds to those of FIG. 2 areindicated by the same symbols and their detailed description is omitted.

A timing generator 182 in a control portion 18 generates a drive signalCRa in accordance with a maximum value of the set frame rate FRs, whichis set via the user interface portion 16, and supplies it to the imagepick-up portion 12. The image pick-up portion 12 generates, based on thedrive signal CRa, an image pick-up signal, i.e., an image pick-up signalSpa having a fixed frame rate FRq higher than the reference frame rateFRr and supplies it to the camera-processing circuit 131 in the signalprocessing portion 17. If the set frame rate FRs can be changed up to n(which is positive) times the reference frame rate FRr, the imagepick-up portion 12 generates an image pick-up signal Spa having n timesthe reference frame rate FRr and supplies it to the camera-processingcircuit 131. That is, the image pick-up portion 12 generates the imagepick-up signal Spa having a fixed frame rate without being influenced bythe set frame rate FRs which is set via the user interface portion 16.

Further, the timing generator 182 generates a timing signal CTasynchronized with the drive signal CRa and supplies it to thecamera-processing circuit 131, the audio-processing circuit 132, and avalid frame signal generation circuit 183 in the signal processingportion 17.

The camera-processing circuit 131 supplies a valid data picking circuit171 with image data DVa having the fixed frame rate FRq generated on thebasis of the image pick-up signal Spa. The audio-processing circuit 132supplies the valid data picking circuit 171 with audio data DAagenerated by performing sampling based on the timing signal CTa having aconstant frequency.

The image pick-up control circuit 181 generates a set information signalCF indicating a set frame rate FRs based on the operation signal PSafrom the user interface portion 16 and supplies it to the valid framesignal generation circuit 183.

The valid frame signal generation circuit 183 generates an extractioncontrol signal CC for extracting data from the image data DVa by theframe unit to generate image data DV having the set frame rate FRs,based on a ratio between a frame rate FRq that is a constant value ofthe image data DVa and a set frame rate FRs indicated by the setinformation signal CF. Furthermore, the valid frame signal generationcircuit 183 supplies the valid data picking circuit 171 with thisextraction control signal CC in synchronization with the timing signalCTa. For example, if a frame rate FRq of the image data DVa is n timesthe reference frame rate FRr and a set frame rate FRs is (n/2) of thereference frame rate FRr, the valid frame signal generation circuit 183generates the extraction control signal CC for extracting data from theimage data DVa every other frame by frame unit and supplies it to thevalid data picking circuit 171 in synchronization with the timing signalCTa.

Further, the valid frame signal generation circuit 183 generates framerate information DM-FRs indicating a set frame rate FRs based on the setinformation signal CF and supplies it to the output portion 15.Furthermore, since the number of frames in a frame period having thereference frame rate FRr can be identified by the extraction controlsignal CC, the valid frame signal generation circuit 183 sets asub-frame number BN to each of the frames in each frame period havingthe reference frame rate FRr and supplies this sub-frame number BN alsoto the output portion 15 as frame identification information DM-BN.

The valid data picking circuit 171 extracts the image data DVa and thwaudio data DAa of a frame indicated by the extraction control signal CCand supplies them as image data DV and audio data DA respectively to theoutput portion 15. Further, although not shown, the valid frame signalgeneration circuit 183 supplies the valid data picking circuit 171 withframe rate information DM-FR indicating a set frame rate FRs, so thatthe valid data picking circuit 171 may thin out the audio data DAa inaccordance with a ratio between the set frame rate FRs and a frame rateat which the audio data DAa has been generated. For example, if a framerate FRq at which the audio data DAa has been generated is n times thereference frame rate FRr and the set frame rate FRs is (n/2) of thereference frame rate FRr, it thins out the audio data DAa for everyother sample. In this case, a thinning-out interval can be made smallerthan that in the case of thinning out the audio data by frame unit, sothat a sound based on the audio data DA can be provided with a bettersound quality.

In such a manner, by making the frame frequency of the image data DVaconstant, it becomes unnecessary to vary an operating frequency at theimage pick-up portion 12 or the camera-processing circuit 131 in thesignal-processing portion 17, thereby simplifying a configuration of theimage pick-up portion 12 and that of the camera-processing circuit 131.Further, only by extracting data from the image data DVa by frame unit,image data DV having a set frame rate FRs can be generated, so that theimage data DV having a desired set frame rate FRs can be easilygenerated from the image data DVa.

Further, the image pick-up apparatus may be provided with an imagememory or an adder and a divider to add up image data for eachpredetermined number of frames, thereby generating the image data DV. Inthis case, a variable range of the frame rate of the image signal Sp canbe made smaller. That is, by adding up n number of frames of the imagesignal Sp and dividing a signal level by n, it is possible to obtain asignal having the frame rate (1/n) even if the frame rate of the imagepick-up signal Sp is not divided by n.

FIGS. 5A-5E and FIGS. 6A-6E are explanatory illustrations ofrelationships between the image data DV generated at the image pick-upapparatuses 10 and 10 a and the accessory information DM. As shown inFIG. 5A, by making a set frame rate FRs equal to the reference framerate FRr or twice it, to image data DV shown in FIG. 5B (in this figure,a frame image based on the image data DV is shown), accessoryinformation DM which contains frame rate information DM-FRs of FIG. 5Cshowing the set frame rate FRs and frame identification informationDM-BN of FIG. 5D showing the sub-frame number BN is linked. It is to benoted that FIG. 5E shows a relationship between a lapse of time and aframe image. Further, the frame rate information DM-FRs may indicate notonly the set frame rate FRs but also a magnification of the set framerate FRs with respect to the reference frame rate FRr. The frame rateinformation DM-FRs shown in FIG. 5C and the subsequent indicates themagnification thereof.

If a set frame rate FRs is made equal to the reference frame rate FRr orhalf that, as shown in FIG. 6A, to image data DV shown in FIG. 6B (inthis figure, a frame image based on the image data DV is shown),accessory information DM which contains frame rate information DM-FRs ofFIG. 6C showing the set frame rate FRs and frame rate information DM-BNof FIG. 6D showing the sub-frame number BN is linked. FIG. 6E shows arelationship between a lapse of time and a frame image.

The following will describe the editing apparatus 30. FIG. 7 shows aconfiguration of the editing apparatus 30. Materials-data DTm suppliedto the editing apparatus 30 is supplied to an information detectioncircuit 311 in a materials-take-in portion 31. The information detectioncircuit 311 detects accessory information DM from the materials-dataDTm. This detected accessory information DM is supplied to a data-basingprocessing circuit 312. Further, image data DV and audio data DAcontained in the materials-data DTm are supplied to the data-basingprocessing circuit 312.

The data-basing processing circuit 312 correlates the image data DV andthe audio data DA with the accessory information DM detected by theinformation detection circuit 311 and stores them in a data storage 321in an edit processing-portion 32. Further, based on the accessoryinformation DM stored in the data storage 321 and the image data DV andthe audio data DA correlated with this accessory information DM, thedata-basing processing circuit 312 generates database information DBthat enables details of the materials-data to be easily confirmed andsupplies it to an edit control portion 33. For example, the databaseinformation DB comprises information for enabling details ofmaterials-data (e.g., thumbnail) to be identified, time length of thematerials-data, a set frame rate FRs, a sub-frame number BN, andinformation such as storage locations in the data storage 321.

The edit control portion 33 generates image data DVg for enabling editprocessing to be carried out in a GUI (Graphical User Interface)environment and image data DVi for displaying details of databaseinformation and supplies them to an image output signal generationcircuit 351. The image output signal generation circuit 351 generates animage signal Svm based on the supplied image data pieces DVg and DVi andoutputs it to the edited-image display 40. By thus supplying the imagesignal Svm to the edited-image display 40, it is possible to displaywhat-like materials-data is stored etc. on a screen of the edited-imagedisplay 40.

Further, the edit control portion 33 controls post-productionprocessing. That is, a user interface portion 34 connected to the editcontrol portion 33 supplies an operation signal PSe that utilizesdisplay in the GUI environment, so that if the operation signal PSeinstructs to select any materials-data, the edit control portion 33generates a read control signal RC in accordance with this operationsignal PSe and supplies it to a read/write processing circuit 322 in anedit processing portion 32. Further, if the operation signal PSeinvolves an edit operation such as working or combination of readmaterials-data, the edit control portion 33 generates an edit controlsignal ET in accordance with the operation signal PSe and supplies it toa signal edit circuit 323 in the edit processing portion 32.Furthermore, when editing of the materials-data is finished to completecontents-data and if the operation signal PSe indicates an operation tostore the contents-data in the data storage 321, the edit controlportion 33 generates a write control signal WC in accordance with theoperation signal PSe and supplies it to the read/write processingcircuit 322. Further, if the operation signal PSe indicates output ofthe contents-data, the edit control portion 33 generates an outputcontrol signal RP in accordance with the operation signal SPe andsupplies it to the read/write processing circuit 322. If the operationsignal PSe regulates a speed range for reproduction of thecontents-data, the edit control portion 33 generates a speed rangesetting signal LP in accordance with the operation signal PSe andsupplies it to the signal edit circuit 323.

The read/write processing circuit 322 reads requested materials-datafrom the data storage 321 based on the read/write control signal RC andsupplies it to the signal edit circuit 323. Further, the read/writeprocessing circuit 322 stores the completed contents-data DC in the datastorage 321 based on the write control signal WC. Further, theread/write processing circuit 322 reads requested contents-data DC fromthe data storage 321 based on the output control signal RP and outputsit.

The signal edit circuit 323 performs edit processing such as processing,combining, and deleting of images and audio based on the edit controlsignal ET, using the image data DV and/or the audio data DA contained inthe materials-data read from the data storage 321. In this process, thesignal edit circuit 323 supplies the image output signal generationcircuit 351 with image data DVe which is before or after being edited orbeing edited and supplies an audio output signal generation circuit 352with audio data DAe which is before or after being edited or beingedited. Further, when altering a frame rate of image data DV or audiodata DA in edit processing, the signal edit circuit 323 alters accessoryinformation DM also in such a manner that it may match the image dataand the audio data as edited. Furthermore, the signal edit circuit 323generates contents-data DC by interlinking edited image data DV or audiodata DA and accessory information DMc that includes frame rateinformation DM-FRs indicating set frame rate FRs corresponding to theedited image data DV or audio data DA and frame identificationinformation DM-BN. When supplied with the speed range setting signal LP,the signal edit circuit 323 links also speed range information thatindicates a reproduction speed range of the contents-data DC as theaccessory information DMc based on this speed range setting signal LP.Furthermore, the signal edit circuit 323, when receiving a title and arecommended reproduction speed of contents from the user interfaceportion 34, links such the information as the accessory information DMc.Further, when obtaining reproduction time length information of thecontents-data in edit processing, this information may also be linked asthe accessory information DMc. Furthermore, when receiving a maximumpossible reproduction speed of contents-data, this maximum speed is alsolinked as the accessory information DMc. Further, no sub-frame number BNis added to materials-data, the signal edit circuit 323 or the editcontrol portion 33 performs the above-mentioned processing shown in FIG.3, to set a sub-frame number BN, thereby providing frame identificationinformation DMc-BN.

The image output signal generation circuit 351 in the edit output signalgeneration portion 35, as described above, generates an image signal Svmbased on image data DVg and DVi supplied from the edit control portion33 and supplies it to the edited-image display 40. Therefore,information concerning materials-data can be displayed in the GUIenvironment. Furthermore, the image ID output signal generation circuit351 generates the image signal Svm based on the image data DVe suppliedfrom the signal edit circuit 323. Accordingly, the user can confirmimages which are before or after being edited or being edited, on thescreen of the edited-image display 40.

The audio output signal generation circuit 352 converts audio data DAesupplied from the signal edit circuit 323 into an analog audio signalSam and supplies it to the edited-audio output apparatus 41 constitutedof, for example, a speaker or headphone according to a desired signallevel. Accordingly, the user can confirm sounds which are before orafter being edited or being edited according to a sound output from theedited-audio output apparatus 41.

When, in such a manner, post-production processing by use of thematerials-data DTm is performed at the editing apparatus 30 to completecontents-data DC, this completed contents-data DC is supplied to thecontents-transmission apparatus 50 and then, from thiscontents-transmission apparatus 50 it is supplied to thecontents-reproduction apparatus 70 of the user.

FIG. 8 shows a configuration of the contents-transmission apparatus 50.The transmit contents-data DC supplied from the editing apparatus 30 issupplied to a write-processing portion 51. The write-processing portion51, which is connected to a contents-accumulation apparatus 521 in atransmit data generation portion 52, stores the supplied transmitcontents-data in the contents-accumulation apparatus 521. It is to benoted that contents-data DC is not limited to that supplied from theediting apparatus 30; for example, materials-data etc. generated by theimage apparatus 10 may be used as the contents-data DC.

The transmit data generation portion 52 is adapted to generate transmitdata DTz based on the contents-data DC and includes thecontents-accumulation apparatus 521; to which a read-processing circuit522 is connected. This read-processing circuit 522 receives bandinformation WB of a transmission channel at the time of transmission ofthe transmit contents-data and contents-request signal RQ from the sideof the contents-reproduction apparatus from a transmission-processingportion 53, described later.

The read processing circuit 522, based on the band information WB andthe accessory information DMc of requested contents-data accumulated inthe contents-accumulation apparatus 521, adjusts a frame rate bycontrolling reading of the requested contents-data and supplies aninformation modification circuit 523 with the contents-data DCza afterthe frame rate is adjusted.

For example, if a quantity of one frame of data is BD number of bitswhen encoding processing is performed by a later-described encoder 524and a set frame rate FRs indicated by the frame rate information DMc-FRsis n (which is positive) times the reference frame rate FRr, an amountof data BT transmitted in a unit time is “BT=BD×n×FRr+BH”. It is to benoted that the amount of data BH is given as a quantity that includesheader information etc. added when the contents-data is transmitted in apacket.

If, in this case, an amount of transmittable data BA (bandwidth)indicated by the band information WB is not smaller than the amount ofdata BT, the frame rate of the contents-data is not adjusted, so thatthe contents-data is sequentially read from the contents-accumulationapparatus 521 and supplied to the information modification circuit 523.If the bandwidth BA is smaller than the amount of data BT, on the otherhand, the frame rate adjustment is performed on image data etc. in thecontents-data, to decrease the amount of data so that an image or asound may not be interrupted during a streaming operation forreproducing the transmit data with it being received. For example, froma set frame rate FRs indicated by the accessory information DMc and thereference frame rate FRr, a multiple “m” of the set frame rate FRs withrespect to the reference frame rate FRr is identified. Furthermore,divisors of the identified multiple “m” are obtained, so that a maximumvalue of the divisors except “m” and the reference frame rate FRr aremultiplied by each other to provide a set frame rate after adjustment.That is, since the maximum divisor value is “5” when “m=10”, such framerate adjustment that “m=5” is performed. In this frame rate adjustment,contents-data in every other frame, that is, the frames having evensub-frame numbers of “0, 2, 3, 6, 8” is read by utilizing the frameidentification information DMc-BN, thereby generating contents-datahaving a frame rate five times the reference frame rate FRr. If “m=9”,for example, such frame rate adjustment that “m=3” is performed, at aninterval of two frames, that is, contents-data in the frames havingsub-frame numbers of “0, 3, 6” is read by utilizing the frameidentification information DMc-BN, thereby generating contents-dataafter having an adjusted frame rate. Further, if the amount of data BTafter adjustment is larger than the bandwidth BA, further frame rateadjustment is performed. In such a manner, if the maximum value of thedivisors except “m” is used to determine a frame rate after adjustment,only by performing thinning-out operation for each frame utilizing theframe identification information DMc-BN when reading contents-data, thecontents-data with the frame rate as adjusted can be generated easily.

Then, if the amount of data BT after adjustment is larger than thebandwidth BA even with “m=1”, such a frame thinning-out operation that“m−1/k” (k: natural number) can be performed, thereby further reducingthe amount of data BT. Further, if the bandwidth BA is changed, theframe rate is varied in accordance with this change in bandwidth BA.

Further, a sample thinning-out operation is performed on the audio dataof the contents-data in accordance with frame rate adjustment for theimage data, so that the audio data having an adjusted frame rate can begenerated. For example, if the image data is read for every other frame,the audio data is read for every other sample. If the image data is readat an interval of two frames, the audio data is read at an interval oftwo samples, thereby generating the audio data having an adjusted framerate.

When frame rate adjustment is performed by the read-processing circuit522, the information modification circuit 523 modifies accessoryinformation DMza of contents-data DCza so that it may match the adjustedframe rate, thereby providing accessory information DMz that indicates aframe rate properly. Furthermore, contents-data DCz to which thisaccessory information DMz is linked is supplied to the encoder 524. Forexample, if “m=10” is adjusted to “m=5”, the set frame rate FRs isaltered from “×10” to “×5” by modification, so that frame rateinformation DMza-FRs indicating that the set frame rate FRs is “×10” ischanged to frame rate information DMz-FRs indicating that the set framerate FRs is “×5”. Corresponding to this change in the set frame rateFRs, the frame identification information DMza-BN is also changed. Thatis, it is changed to such frame identification information DMz-BN thatsub-frame numbers BN “0-9” are replaced by sub-frame numbers BN “0-4”.Furthermore, using these frame rate information DMz-FRs and frameidentification information DMz-BN after being changed, the accessoryinformation DMc is changed to the accessory information DMz.

The encoder 524 encodes the image data DVz and the audio data DAz of thesupplied contents-data DCz into a signal suitable for transmission,thereby generating encoded data DZ. For example, using an encodingsystem standardized as MPEG (Moving Picture Experts Group) 4, they areencoded into a signal suitable for streaming transmission. To theencoded data DZ obtained by this encoding processing, the accessoryinformation DMz is linked and they are supplied as transmit data DTz tothe transmission-processing portion 53. In such a manner, by performingencoding processing, it is possible to transmit contents-dataefficiently.

When requested for contents-data by a transmit signal TMrq supplied fromthe contents-reproduction apparatus 70, the transmission-processingportion 53 supplies the read-processing circuit 522 with thecontents-request signal RQ that indicates the requested contents-data.Further, the transmission-processing portion 53 generates the bandinformation WB concerning a band of the transmission channel 60 andsupplies it to the read-processing circuit 522. Furthermore, based onthe request for the contents-data, the transmission-processing portion53 supplies the transmit data DTz supplied from the encoder 524 as atransmit signal in accordance with a predetermined protocol to thecontents-reproduction apparatus 70, which has requested for thecontents-data, through the transmission channel 60.

As the band information WB to be supplied to this read-processingcircuit 522, traffic information can be used that can be obtained from amanagement information base (MIB) of a network appliance such as arouter that constitutes the transmission-processing portion 53. Further,a measuring packet can be transmitted to the contents-reproductionapparatus 70 to identify a band by measuring a response time etc. fromthe contents-reproduction apparatus 70, thereby using a result of thisidentification as the band information WB.

Further, the amount of data BT may be varied in accordance with thebandwidth BA not only by performing frame rate adjustment at theread-processing circuit 522 based on the band information WB but byvarying a data compression ratio at the encoder 524 based on the bandinformation WB. In this case, the amount of data can be controlledfurther finely, so that it is possible to suppress deterioration inquality of images and sounds to be transmitted even if the bandwidth BAis decreased. Furthermore, the amount of data BT can be adjusted at theencoder 524 by making constant an adjusted frame rate irrespective ofthe band information WB during a frame period when the set frame rateFRs stays constant. In this case, it is possible to prevent a portion ofcontents having a desired frame rate set by the image pick-up apparatus10 or the editing apparatus 30 from being adjusted to a different framerate in accordance with the bandwidth BA.

Furthermore, if a recommended reproduction speed is set to the accessoryinformation DMc, frame rate adjustment may be performed within a rangein which reproduction is possible at the recommended reproduction speed,to adjust the amount of data BT at the encoder 524 if the number offrames needs to be made smaller than that at the time of reproduction atthe recommended reproduction speed. In this case, contents can bereproduced at the recommended reproduction speed even if the bandwidthof the transmission channel 60 is decreased.

Incidentally, the contents-transmission processing performed by theabove-mentioned contents-transmission apparatus 50 can be realized alsoby software processing by use of a computer. A configuration employed inthe case of contents-transmission by means of software processing isshown in FIG. 9.

As shown in FIG. 9, the computer has a built-in CPU (Central ProcessingUnit) 551, and to CPU 551 via a bus 560 an ROM 552, an RAM 553, a dataaccumulation portion 554 constituted of a mass-capacity hard disk driveetc., and an input/output interface 555 are connected. Further, to theinput/output interface 555, a signal input portion 561, a communicationportion 562, and a recording medium drive 563 are connected.

The CPU 551 executes programs stored in the ROM 552, the RAM 553, or thedata accumulation portion 554, thereby performing contents-transmissionprocessing. Contents-data, which is input to the signal input portion561, is stored in the data accumulation portion 554 via the input/outputinterface 555 and the bus 560. Further, when being supplied with thecontents request signal RQ via the communication portion 562, the CPU551 reads requested contents-data from among the contents-data stored inthe data accumulation portion 554 and controls such the reading toadjust a frame rate so that the contents-data may have an amount of datathat matches a capacity of the transmission channel 60. Furthermore, theCPU 551 generates transmit data DTz by performing encoding suitable fortransmission. The transmit data DTz thus generated is output through thecommunication portion 562.

It is to be noted that a program used to transmit contents may be storedbeforehand in the ROM 552 or the data accumulation portion 554 or theprogram used to transmit contents may be recorded in a recording mediumby the recording medium drive 563 or the program recorded in therecording medium may be read and executed by it. Furthermore, theprogram may be transmitted or received by the communication portion 562through a wireline or wireless transmission channel so that the receivedprogram can be executed by a computer.

FIG. 10 is a flowchart for showing a contents-transmission processingoperation. At step ST11, the CPU 551 takes in contents-data DC andstores the contents-data DC as input to the signal input portion 561 inthe data accumulation portion 554. It is to be noted that thecontents-data is not limited to that supplied from the editing apparatus30; materials-data etc. generated by the image pick-up apparatus 10 maybe stored as contents-data in the data accumulation portion 554.

At step ST12, the CPU 551 decides whether contents-data is requested. Ifno contents-data is requested, the process returns to step ST12 and, ifcontents-data is requested through, for example, the communicationportion 562, the process goes to step ST13.

At step ST13, the CPU 551 reads accessory information of the requestedcontents-data and the process goes to step ST14.

At step ST14, the CPU 551 detects a band of the transmission channeland, in accordance with the detected band, controls the contents-data tobe read from the data accumulation portion 554 utilizing the frameidentification information, thereby adjusting the frame rate.

At step ST15, the CPU 551 modifies the accessory information DMza of theread contents-data DCza so that it may match the adjusted frame rate,thereby providing accessory information DMz. At step ST16, the CPU 551performs encoding processing suited to the transmission channel usingthe contents-data DCz having the modified accessory information, therebygenerating encoded data DZ. Furthermore, it generates transmit data DTzusing the encoded data DZ thus generated and the accessory informationDMz thus modified and the process goes to step ST17.

At step ST17, the PCU 551 outputs the transmit data DTz thus generatedfrom the communication portion 562 toward a destination of the requestedcontents-data.

Next, the following will describe the contents-reproduction apparatus.FIG. 11 shows a configuration of the contents-reproduction apparatus 70.A transmit signal TMz supplied from the contents-transmission apparatus50 is supplied to a communication circuit 711 in an input portion 71.The input portion 71 is adapted to take in contents-data, while thecommunication circuit 711 in the input portion 71 generates transmitdata DTz from the transmit signal TMz thus supplied and extracts theencoded data DZ and the accessory information DMz from this transmitdata DTz. Furthermore, the communication circuit 711 supplies theextracted accessory information DMz to an information storage circuit712 and the encoded data DZ to a data-holding circuit 713. Further, thecommunication circuit 711 generates the transmit signal TMrq based onthe contents request signal RQ from a later-described reproductioncontrol portion 72 and supplies it to the contents-transmissionapparatus 50.

The information storage circuit 712 stores the supplied accessoryinformation DMz. The data-holding circuit 713 stores the encoded data DZthus supplied.

To the reproduction control portion 72, a user interface portion 73 isconnected. If an operation signal PSp from the user interface portion 73requests for contents-data, the reproduction control portion 72generates the contents request signal RQ based on the operation signalPSp and supplies it to the communication circuit 711, thereby requestingthe contents-transmission apparatus 50 to transmit the contents-data.

Further, if the operation signal PSp instructs to reproducecontents-data, the reproduction control portion 72 supplies a readcontrol signal CN to the data-holding circuit 713 to read from thedata-holding circuit 713 the encoded data DZ of contents instructed tobe reproduced and supplies it to a reproduction processing portion 74.Furthermore, the reproduction control portion 72 reads from theinformation storage circuit 712 the accessory information DMz thatcorresponds to the encoded data DZ thus read, generates image data DVsfor displaying information contained in the accessory information DMz,for example, such image data as to indicate reproduction-enabling speedrange based on restriction information contained in the accessoryinformation DMz or if this accessory information DMz contains timeinformation such as a time code, image data that indicates a total sumlapse of time, a moment of a reproduction position, etc. indicated bythis time information, and supplies it to the reproduction processingportion 74. Accordingly, the reproduction-enabling speed range, thetotal sum lapse of time, the moment of the reproduction position, etc.are displayed on a screen of the contents-presentation apparatus 80 suchas a TV set or a monitor apparatus. Further, if the accessoryinformation DMz contains no speed range information, areproduction-enabling speed range is set as described for theabove-mentioned editing apparatus 30. The reproduction-enabling speedrange thus set is displayed on the screen of the contents-presentationapparatus 80. If the operation signal PSp instructs to vary acontents-reproduction speed FP, the reproduction control portion 72generates a presentation control signal CP for controlling operations ofthe reproduction processing portion 74 based on the accessoryinformation DMz and supplies it to the reproduction processing portion74.

Further, if a maximum reproduction-enabling speed of contents isindicated by the accessory information DMz, the reproduction controlportion 72 sets a maximum speed in a variable range of the reproductionspeed FP as the maximum speed indicated by the accessory informationDMz. Furthermore, in a case where a recommended reproduction speed isindicated by the accessory information DMz, if no reproduction speed isindicated by the operation signal PSp, the reproduction control portion72 generates the presentation control signal CP so that reproduction maybe performed at this recommended reproduction speed. It is to be notedthat if a title or a time length of contents is indicated by theaccessory information DMz, the reproduction control portion 72 displaysthese pieces of information on the screen of the contents-presentationapparatus 80.

The reproduction processing portion 74 for reproducing contents at avariable speed decodes the encoded data DZ supplied from thedata-holding circuit 713, to generate image data DVz and audio data DAzof the contents. The reproduction processing portion 74 further performsthinning-out or repeating processing by use of the frame identificationinformation DMz-BN on the generated image data DVz and audio data DAzbased on the presentation control signal CP, to generate an image signalSvz and an audio signal Saz in accordance with a reproduction speed FPthat is set by the user or equal to the recommended reproduction speedand supplies them to the contents-presentation apparatus 80, therebypresenting the contents. Further, when being supplied with the imagedata DVs that indicates a variable range of the reproduction speed FP,the reproduction processing portion 74 generates the image signal Svzfor displaying a variable range of this reproduction speed FP on thescreen of the contents-presentation apparatus 80.

It is to be noted that if the encoded data DZ is composed of intra-frameencoded data, the reproduction processing portion 74 may, based on thepresentation control signal CP from the data-holding circuit 713, readthe encoded data DZ by thinning it out by a frame unit. In this case, itis unnecessary to decode the thinned out portion of the image data,thereby enabling decoding processing to be easily performed.

Further, the contents-reproduction apparatus 70 may use a recordingmedium in which contents-data is recorded. In this case, the data can beprocessed similarly by separating the accessory information DMz and theencoded data DZ from a reproduction signal generated by reproducing therecording medium, storing this accessory information DMz in theinformation storage circuit 712, and storing the encoded data DZ in thedata-holding circuit 713.

Incidentally, the above-mentioned contents-reproduction processing bythe contents-reproduction apparatus 70 can be realized by softwareprocessing executed by a computer. A configuration ofcontents-reproduction by means of this software processing is shown inFIG. 12.

As shown in FIG. 12, the computer has a built-in CPU 751, to which anROM 752, an RAM 753, a data accumulation portion 754, and aninput/output interface 755 are connected via a bus 760. Furthermore, tothe input/output interface 755, a communication portion 761, a userinterface portion 762, a signal output portion 763, and a recordingmedium drive 764 are connected.

The CPU 751 executes programs stored in the ROM 752, the RAM 753 or thedata accumulation portion 754, thereby performing contents-transmissionprocessing based on the operation signal PSp from the user interfaceportion 762. In this case, if being supplied with transmit data DTz, thecommunication portion 761 extracts the encoded data DZ and the accessoryinformation DMz. The encoded data DZ and the accessory information DMzthus extracted at this communication portion 761 are stored in the dataaccumulation portion 754. Further, the CPU 751 reads or decodes theencoded data DZ stored in the data accumulation portion 754 based on theoperation signal PSp from the user interface portion 762, to generateimage data DVz and audio data DAz and supply them to the signal outputportion 763. The signal output portion 763 generates an image signal Svzand an audio signal Saz suitable for the contents-presentation apparatus80 based on the image data DVz and the audio data DAz and outputs them.

It is to be noted that a program used for contents-reproductionprocessing may be stored beforehand in the ROM 752 or the dataaccumulation portion 754 or a program used for contents-reproductionprocessing may be recorded in a recording medium by the recording mediumdrive 764 or a program recorded in a recording medium may be read andexecuted thereby. Furthermore, the program may be transmitted orreceived by the communication portion 761 through a wireline or wirelesstransmission channel so that the received program can be executed by acomputer.

FIG. 13 is a flowchart for showing a contents-reproduction processingoperation. When contents-data is reproduced, the CPU 751 performs inputoperations to allow an image required to constitute a GUI environment tobe displaying on the contents-presentation apparatus 80 and the userinterface portion 762 to perform operations in accordance with thisdisplayed image.

FIG. 14 shows an example of an image displayed on thecontents-presentation apparatus 80, which displays images for GUI. Onthe screen of the contents-presentation apparatus 80, a viewer portion801 for displaying an image on contents, a speed varying console portion802 serving as an interface for varying the reproduction speed FP, areproduction speed display portion 803 for displaying the reproductionspeed FP, an operation control portion 804 for switching an operationmode, a sound volume, etc., a title display portion 805 for displaying atitle of the contents, a time display portion 806 for displaying areproduction time of the contents and a current time, a reproductionposition display portion 807 for indicating a current reproductionposition, etc. are provided.

At step ST21 of FIG. 13, the CPU 751 reads the accessory information DMzon contents from the data accumulation portion 754 and outputs the imagesignal Svz and the audio signal Saz generated on the basis of theaccessory information DMz to the contents-presentation apparatus 80 fromthe signal output portion 763 via the input/output interface 755.Accordingly, display in accordance with the accessory information DMz isprovided on the contents-presentation apparatus 80. For example, a titleand a time length of the contents are displayed at the title displayportion 805 and the time display portion 806, respectively. Further,based on speed range information, a minimum speed and a maximum speedare displayed at the speed varying console portion 802.

At step ST22, the CPU 751 identifies whether a reproduction startoperation of the contents is performed on the basis of the operationsignal PSp utilizing the operation control portion 804. If noreproduction start operation is performed, the CPU 751 is performed sothat the process returns to step ST22 and, otherwise, goes to step ST23.

At step ST23, the CPU 751 sets reproduction processing conditions inaccordance with a reproduction speed FP and a set frame rate FRs, thatis, determines an interval for data thinning out and the number of datarepeating which are performed when generating the image signal Svz andthe audio signal Saz from the respective image data DVz and audio dataDAz obtained by decoding the encoded data DZ.

At step ST24, the CPU 751 reads the encoded data DZ from the dataaccumulation portion 754 and decodes it to generate the image data DVzand the audio data DAz and, based on the reproduction processingconditions determined at step ST23, thins out or repeats the data byutilizing the frame identification information DMz-BN, therebygenerating the image signal Svz and the audio signal Saz forpresentation of contents. The CPU 751 supplies these generated imagesignal Svz and audio signal Saz to the contents-presentation apparatus80, so that a reproduced image having a reproduction speed FP indicatedby a cursor position (which is expressed in a heavy line) in the speedvarying console portion 802 is displayed in the viewer portion 801 ofthe contents-presentation apparatus 80. Further, the reproduction speedFP used in this case is displayed in the reproduction speed displayportion 803, while a reproduction time and a reproduction position aredisplayed in the time display portion 806 and the reproduction positiondisplay portion 807, respectively. Further, the contents-presentationapparatus 80 outputs a reproduced audio having a reproduction speed FPindicated at the cursor position in the speed varying console portion802.

At step ST25, the CPU 751 identifies whether the reproduction speed FPis changed by moving the cursor position in the speed varying consoleportion 802. If the CPU 751 identifies that the reproduction speed FP ischanged, the process returns to step ST23 and, if the CPU 751 identifiesthat no reproduction speed FP is changed, the process goes to step ST26.

At step ST26, the CPU 751 identifies whether the reproduction operationis finished. If the CPU 751 identifies that no operation is performed tostop reproduction or that the contents-reproduction position is not anend position, the process returns to step ST25. If the stop operation isperformed or if the reproduction position is at the end position, theCPU 751 finishes the speed varying operation.

FIG. 15 is a flowchart for showing an operation of setting reproductionprocessing conditions to an image. At step ST31, the CPU 751 identifiesthe reproduction speed FP based on the cursor position in the speedvarying console portion 802 and the process goes to step ST32. At stepST32, by multiplying the reference frame rate FRr by one and multiplyinga cursor initialization position in the speed varying console portion802 by one, the reproduction speed FP upon start of reproductionoperation is set. Further, if a reproduction speed FP is recommended bythe editing apparatus 30, the CPU 751 sets a position of thisrecommended reproduction speed Fp as the cursor initialization positionand the recommended reproduction speed FP as the reproduction speed FPupon start of reproduction operation. Furthermore, if a cursor positionis moved by the user, the CPU 751 sets a speed in accordance with thecursor position as the reproduction speed FP.

At step ST32, the CPU 751 identifies a set frame rate FRs based on framerate information DMz-FRs contained in the accessory information DMz andthe process goes to step ST33. At step ST33, the CPU 751 multiplies thereproduction speed FP and the set frame rate FRs by each other, therebycalculating an identification value FD for determining reproductionprocessing conditions.

At step ST34, the CPU 751 determines the reproduction processingconditions based on the identification value FD. In this case, if theidentification value FD is not less than one and contains no fractionsbelow decimal point, the CPU 751 determines the reproduction processingconditions so that images may be output after being thinned out at aframe interval in accordance with the identification value FD. If theidentification value FD is not less than one and contains fractionsbelow decimal point, the CPU 751 thins out images by utilizing the frameidentification information DMz-BN at a frame interval in accordance withan integral portion of the identification value FD and, if images areobtained as many as a number of frames that matches a desiredreproduction speed, determines the reproduction processing conditions sothat positions of the images may be moved over to the next initial valueof a sub-frame number BN. If the identification value FD is less thanone, the CPU 751 determines the reproduction processing conditions sothat the same image may be output repeatedly until the number of framesthat matches a desired reproduction speed is reached. Based on the thusdetermined reproduction processing conditions, the processing at stepST24 is performed, to enable presenting an image on contents at adesired reproduction speed properly.

FIGS. 16A-16M show a reproduction operation in a case where theidentification value FD is not less than one and contains no fractionsbelow decimal point. FIG. 16A shows images based on such image data DVzthat its set frame rate FRs may be 10 times the reference frame rateFRr. FIG. 16B shows frame rate information DMz-FRs that indicates a setframe rate FRs of a frame image, FIG. 16C shows frame identificationinformation DMz-BN that indicates a sub-frame number BN of a frameimage, and FIG. 16D shows an absolute frame number AN.

If, in this case, the reproduction speed FP is a multiplied-by-⅕ speed,the identification value FD becomes “10×(⅕)=2”. Accordingly, as shown inFIGS. 16E-16G, at an interval of “FD=2” frames, that is, for every otherframe, image data is used by utilizing the frame identificationinformation DMz-BN to generate an image signal Svz, thereby enabling areproduced image at a multiplied-by-⅕ speed to be display on thecontents-presentation apparatus 80. It is to be noted that FIG. 16Eindicates frame identification information DMz-BN of an image to bedisplayed, FIG. 16F indicates an absolute frame number AN of an image tobe displayed, FIG. 16F indicates frame identification information DMz-BNof an image to be displayed, and FIG. 16G shows a frame image to bedisplayed by the image signal Svz.

If the reproduction speed FP is a multiplied-by-1 speed, theidentification value FP becomes “10×1=10”. Accordingly, as shown inFIGS. 16H-16J, by generating the image signal Svz by using the imagedata DVz at an interval of “FD=10” frames, that is, as skipping nineframes of every 10 frames by utilizing frame identification informationDMz-BN, a reproduced image having a multiplied-by-1 speed can bedisplayed on the contents-presentation apparatus 80. It is to be notedthat FIG. 16H indicates frame identification information DMz-BN of animage to be displayed, FIG. 16I indicates an absolute frame number AN ofan image to be displayed, and FIG. 16J shows a-frame image to bedisplayed by the image signal Svz.

Further, if the reproduction speed FP is a multiplied-by-2 speed, theidentification value FD becomes “10×2=20”. Accordingly, as shown inFIGS. 16K-16M, by generating the image signal Svz by using the imagedata DVz at an interval of “FD=20” frames, that is, as skipping 19frames of every 20 frames by utilizing the frame identificationinformation DMz-BN, a reproduced image having a multiplied-by-2 speedcan be displayed on the contents-presentation apparatus 80. It is to benoted that FIG. 16K indicates frame identification information DMz-BN ofan image to be displayed, FIG. 16L indicates an absolute frame number ANof an image to be displayed, and FIG. 16M shows a frame image to bedisplayed by the image signal Svz.

FIGS. 17A-17M show a reproduction operation in a case where theidentification value FD is not less than one and contains no fractionsbelow decimal point. FIG. 17A shows a frame image in a case where a setframe rate FRs is seven times the reference frame rate FRr. FIG. 17Bshows frame rate information DMz-FRs that indicates a set frame rate FRsof a frame image, FIG. 17C shows frame identification information DMz-BNthat indicates a sub-frame number BN of a frame image, and FIG. 17Dindicates an absolute frame number AN.

If, in this case, the reproduction speed FP is a multiplied-by-⅓ speed,the identification value FD becomes “7×(⅓)=2.33 . . . ”. Therefore, asshown in FIGS. 17E-17G, in accordance with an integral portion of theidentification value FD, the image data DVz is used at an interval oftwo frames, that is, for every other frame by utilizing the frameidentification information DMz-BN. Furthermore, since the number offrames matches the desired reproduction speed, that is, the speed ismultiplied by (⅓), if the image as many as three frames is output in oneframe period having the reference frame rate FRr, the position of theimage data DVz to be used over to the next initial value of thesub-frame number BN is moved. In this case, image signals Svz aresequentially generated using the pieces of image data DVz havingsub-frame numbers BN of “0”, “2”, and “4”, so that a reproduced imagehaving a multiplied-by-⅓ speed can be displayed on thecontents-presentation apparatus 80. It is to be noted that FIG. 17Eindicates frame identification information DMz-BN of an image to bedisplayed, FIG. 17F indicates an absolute frame number AN of an image tobe displayed, and FIG. 17G shows a frame image to be displayed by theimage signal Svz.

FIGS. 18A-18M show a reproduction operation in a case where theidentification value FD is less than one. FIG. 18A shows a frame imagein a case where a set frame rate FRs is (¼) of the reference frame rateFRr. FIG. 18B shows frame rate information DMz-FRs that indicates a setframe rate FRs of a frame image, FIG. 18C shows frame identificationinformation DMz-BN that indicates a sub-frame number BN of a frameimage, and FIG. 18D indicates an absolute frame number AN.

If, in this case, the reproduction speed FP is a multiplied-by-1 speed,the identification value FD becomes “(¼)×1=¼”. Therefore, as shown inFIGS. 18E-18G, by generating the image signal Svz by using the number offrames in accordance with the reproduction speed, that is, by repeatedlyusing the image data DVz four times for each frame, a reproduced imagehaving a multiplied-by-1 speed can be displayed on thecontents-presentation apparatus 80. It is to be noted that FIG. 18Eshows frame identification information DMz-BN of an image to bedisplayed, FIG. 18F shows an absolute frame number AN of an image to bedisplayed, and FIG. 18G shows an image to be displayed by the imagesignal Svz.

If the reproduction speed FP is a multiplied-by-2 speed, theidentification value FD becomes “(¼)×2=½”. Therefore, as shown in FIGS.18H-18J, by generating the image signal Svz by repeatedly using theimage data DVz twice for each frame, a reproduced image having amultiplied-by-2 speed can be displayed on the contents-presentationapparatus 80. It is to be noted that FIG. 18H shows frame identificationinformation DMz-BN of an image to be displayed, FIG. 18I shows anabsolute frame number AN of an image to be displayed, and FIG. 18J showsa frame image to be displayed by the image signal Svz.

If the reproduction speed FP is a multiplied-by-4 speed, theidentification value FD becomes “(¼)×4=1”. Therefore, as shown in FIGS.18K-18M, by generating the image signal Svz by sequentially using theimage data DVz for each frame, a reproduced image having amultiplied-by-4 speed can be displayed on the contents-presentationapparatus 80. It is to be noted that FIG. 18K shows frame identificationinformation DMz-BN of an image to be displayed, FIG. 18L shows anabsolute frame number AN of an image to be displayed, and FIG. 18M showsa frame image to be displayed by the image signal Svz.

In such a manner, by reading image data at a reading interval based on arecording speed and a reproduction speed utilizing frame identificationinformation, an image having a desired reproduction speed can bedisplayed easily.

The following will describe an audio. FIG. 19 is a flowchart for showingan operation of setting reproduction processing conditions to the audio.If audio data DAz is used for each frame, the sound does not continuebetween the frames, so that a discontinuity occurs in sound. Therefore,the sound is reproduced for each sample.

At step ST41, the CPU 751 decides a reproduction speed as in the case ofstep ST31 and the process goes to step ST42. At step ST42, the CPU 571reads a set frame rate FRs as in the case of step ST32 and the processgoes to step ST43. At step ST43, the CPU 751 calculates anidentification value FD as in the case of step ST33 and the process goesto step ST44.

At step ST44, the CPU 751 determines the reproduction processingconditions based on the identification value FD. In this case, if theidentification value FD is not less than one and contains no fractionsbelow decimal point, it determines the reproduction processingconditions so that audio data may be thinned out at a sampling intervalin accordance with the identification value FD. If the identificationvalue FD is not less than one and contains fractions below decimalpoint, it determines the reproduction processing conditions so that theaudio data may be thinned out at a sampling interval in accordance withan integral portion of the identification value FD from a multipleframes of the set frame rate FRs with respect to the reference framerate FRr to thereby read the audio data as much as a reproduction speed.If the identification value FD is less than one, it determines thereproduction processing conditions so that the audio data may be usedrepeatedly until a number of sample frames that matches a desiredreproduction speed is reached. Based on the thus determined reproductionprocessing conditions, the above-mentioned processing at step ST24 isperformed, to enable a sound on the contents at a desired reproductionspeed to be properly presented.

FIGS. 20A-20E show an audio reproduction operation in a case where theidentification value FD is not less than one and contains no fractionsbelow decimal point. FIG. 20A shows an absolute frame number AN, FIG.20B shows frame rate information DMz-FRs that indicates a set frame rateFRs of a frame image, FIG. 20C shows frame identification informationDMz-BN that indicates a sub-frame number BN of a frame image.

If, in this case, the reproduction speed FP is a multiplied-by-⅕ speed,the identification value FD becomes “10×(⅕)=2” because the set framerate FRs is supposed to be 10 times the reference frame rate FRr.Accordingly, at an interval of “FD=2” samples, that is, for every othersample, the audio data DAz is used to generate an audio signal Saz,thereby enabling a reproduced sound having a multiplied-by-1 speed to beoutput from the contents-presentation apparatus 80. It is to be notedthat FIG. 20D shows a frame which is used to generate the image signalSvz and FIG. 20E shows audio data which is used in the audio signal Sazif the audio data DAz has 14 samples/frame.

FIGS. 21A-21E show an audio reproduction operation in a case where theidentification value FD is not less than one and contains fractionsbelow decimal point. FIG. 21A shows an absolute frame number AN, FIG.21B shows frame rate information DMz-FRs that indicates a set frame rateFRs of a frame image, and FIG. 21C shows frame identificationinformation DMz-BN that indicates a sub-frame number BN of a frameimage.

If, in this case, the reproduction speed FP is a multiplied-by-⅓ speed,the identification value FD becomes “7×(⅓)=2.3 . . . ” because the setframe rate FRs is supposed to be seven times the reference frame rateFRr. Further, if the audio data DAz has 14 samples/frame, the number ofsamples per frame at a multiplied-by-⅓ speed is “14× 3/7=6”.Accordingly, in accordance with an integral portion of theidentification value FD, the audio data DAz is output at an interval oftwo samples, that is, for every other sample and, if the audio data DAzof six samples, which constitute one frame, is output, the audio dataDAz skips to the beginning of the next frame and is output for everyother sample. By thus selecting the audio data DAz and outputting it, areproduced sound having a multiplied-by-⅓ speed can be obtained.Further, if filtering processing is performed in a case where a sound isoutput on the basis of the audio signal Saz, a good reproduced sound canbe output by suppressing an influence due to thinning-out of the audiodata DAz. Furthermore, samples are skipped fixedly in accordance with aset frame rate FRs and a reproduction speed FP to match the number ofsamples at the end of a frame, thereby enabling the audio signal Saz tobe easily output in accordance with a reproduction speed. It is to benoted that FIG. 21C shows a frame which is used to generate the imagesignal Svz and FIG. 21D shows audio data which is used in the audiosignal Saz when the audio data DAz has 14 samples/frame.

Further, in the case of generating the audio signal Saz by thinning outthe audio data DAz, to prevent a reproduced sound to be discontinuousdue to an increase in interval between items of the audio data DAz, theitems of audio data which are used to generate the audio signal Saz maybe thinned out so that they may have a roughly constant interval. Forexample, if a set frame rate FRs is KA times the reference frame rateFRr and a reproduction speed FP is a multiplied-by-1/KB speed, the audiodata is taken out as many as KB number of samples at a roughly constantinterval from the consecutive KB number of samples of the audio data DAzand, based on this taken out audio data, the audio signal Saz isgenerated. It is thus possible to output a reproduced sound having afurther better sound quality, although the processing becomescomplicated as compared to the case shown in FIGS. 21A-21E.

In a case where the identification value FD is less than one, althoughnot shown, audio data is sequentially used repeatedly as many times asthe number of repetitions of frames of an image, thereby enabling theaudio data DAz having a desired reproduction speed to be generated.

In such a manner, a contents-transmission side transmits contents-dataDCz in which the accessory information DMz including frame rateinformation and frame identification information for identifying framesincluded in a reference frame period is linked to main data indicatingan image and/or a sound. Further, a contents-reproduction sidereproduces the image and/or the sound by varying a reproduction speedutilizing the accessory information DMz that contains the frame rateinformation and the frame identification information. Accordingly, theuser not only can view an image etc. having a predetermined reproductionspeed as in the case of a broadcast program but also can view the imageetc. at his desired reproduction speed. For example, by generatingcontents such as a relayed sports program at a set frame rate FRs higherthan the reference frame rate FRr, the user need not wait until aslow-speed reproduction image is supplied from a contents-provider as inthe case of a conventional broadcast program but can view only a desiredscene at a slow reproduction speed while viewing the image at amultiplied-by-1 speed usually.

Further, on the contents-transmission side, a frame rate is adjusted inaccordance with a band of a transmission channel by utilizing the frameidentification information, thereby enabling frame rate adjustment to beeasily executed. Further, on the contents-reproduction side, data canbe, for example, thinned out for each frame easily by utilizing theframe identification information, thereby varying thecontents-reproduction speed easily.

INDUSTRIAL APPLICABILITY

As described above, the present invention is useful in the case oftransmitting image contents etc. and reproducing them and well suitedparticularly to a case where a frame rate at the time of reproduction isvaried.

1-17. (canceled)
 18. A reproduction apparatus comprising: setting meansfor, based on information of a frame rate of transmit data composed bylinking to main data representing an image and/or audio accessoryinformation including information of a frame rate of this main data,setting a variable reproduction range indicating a range of the framerate of the main data to be reproduced; and reproduction means forreproducing the main data at a frame rate in the variable reproductionrange.
 19. The reproduction apparatus according to claim 18, wherein ifthe accessory information includes information indicating a recommendedreproduction speed of the main data and a user avoids specifying areproduction speed thereof, the reproduction means reproduces the maindata at the recommended reproduction speed.
 20. The reproductionapparatus according to claim 18, wherein if the accessory informationincludes information indicating a reproduction-enabling maximum speed ofthe main data, the setting means sets the variable reproduction speedrange using the information that indicates this maximum speed.
 21. Thereproduction apparatus according to claim 18, wherein the accessoryinformation includes frame identification information of each frameincluded in a reference frame period and the reproduction means performsthinning-out or repeating processing on the main data utilizing theframe identification information, thereby making a reproduction speed ofthe main data variable.
 22. A reproduction method comprising: a settingstep for, based on information of a frame rate of transmit data composedby linking to main data representing an image and/or audio accessoryinformation including information of a frame rate of this main data,setting a variable reproduction range indicating a range of the framerate of the main data to be reproduced; and a reproduction step forreproducing the main data at a frame rate in the variable reproductionrange.
 23. The reproduction method according to claim 22, wherein in thereproduction step, if the accessory information includes informationindicating a recommended reproduction speed of the main data and a useravoids specifying the reproduction speed thereof, the main data isreproduced at the recommended reproduction speed.
 24. The reproductionmethod according to claim 22, wherein in the setting step, if theaccessory information includes information indicating areproduction-enabling maximum speed of the main data, the variablereproduction speed range is set using the information that indicatesthis maximum speed.
 25. The reproduction method according to claim 22,wherein the accessory information includes frame identificationinformation of each frame included in a reference frame period and inthe reproduction step, thinning-out or repeating processing is performedon the main data utilizing the frame identification information, therebymaking a reproduction speed of the main data variable.
 26. A program forcausing a computer to perform a reproduction method, the methodcomprising: a setting step for, based on information of a frame rate oftransmit data composed by linking to main data representing an imageand/or audio accessory information including information of a frame rateof this main data, setting a variable reproduction range indicating arange of the frame rate of the main data to be reproduced; and areproduction step for reproducing the main data at a frame rate in thevariable reproduction range. 27-28. (canceled)